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Creativity and Creative Teams 

Richard M. Wood, Steven X. S. Bauer, and Craig A. Hunter 


ABSTRACT 

A review of the linkage between knowledge, 
creativity, and design is presented and related to the 
best practices of multidisciplinary design teams. The 
discussion related to design and design teams is 
presented in the context of both the complete 
aerodynamic design community and specifically the 
work environment at the NASA Langley Research 
Center. To explore way s to introduce know ledge and 
creativity into the research and design environment at 
NASA I singles Research Center a creative design 
activity was executed within the context of a national 
product development activity. The success of the 
creative design team activity gave rise to a need to 
communicate the experience in a straightforward and 
managed approach. As a result the concept of 
creative potential was formulated and assessed w ith a 
survey of a small portion of the aeronautics research 
staff at NASA I^angley Research ( enter. The final 
section of the paper provides recommendations for 
future creativ e organizations and work environments. 


INTRODUCTION 

To design is to create; which requires knowledge, or 
to quote Kucheman. "Ideas and concepts come out of 
the mind, not out of computers or wind tunnels" ‘. 

Design: a new concept, model, or artifact with 

desired function, performance , or end 

Create: to bring something new into being 
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In pursuit of reduced cost the aeronautics community 
has increased its reliance upon computational tools 
for both aerodynamic analysis and design - \ 
Computational based design and decision making also 
dominate the multi-disciplinary design environment. 
As this trend continues, the knowledge that is the 
critical element in design practice could erode as 
computational tools replace the need for human 
interaction, resulting in a stagnant design technology 
landscape. While many still believe that 
computational design is the pathway to improved 
efficiency and effectiveness in the design 
environment, others arc starting to question the 
reliance upon computational tools. A portion of the 
design community believes that it is critical to 
maintain human involvement and learning in the 
design and analy sis process 4 M . 

The literature on design shows a wide diversity of 
methods, concepts, and ideas being investigated and 
developed 1 In general, these concepts fall into two 
groups, those that are computationally -based such as 
Multi-Disciplinary Optimization 4 , and those that am 1 
know ledge- based such as Decision-Based-Design 14 . 
The primary assumption of computationally-based 
design is that a design process can be modeled in the 
computer to allow the computer to find the optimum 
design more efficiently than a human can. The 
assumption behind the computationally -based design 
is based upon the use of explicit and critical 
know ledge only , and does not recognize the role of 
tacit and intuitive know ledge and other human senses 
and capabilities in the design process. 

At the other end of the design spectrum ac- 
knowledge- based design methods 24 in which it is 
recognized that all knowledge as well as passion is 
required to find the best design. Knowledge-based 
design employ s tools that allow a designer to utilize 
their skills, senses, and knowledge in pursuit of a 
desired outcome. Knowledge-based design should not 
he confused with Artificial Intelligence or Hxpert 
Systems, because those systems assume that human 
knowledge and decisions can by fully modeled into a 
process and computerized. It is unlikely that we w ill 
ever be able to totally model the extent of our 
knowledge, because the extent of it is unknown. 
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This paper is designed to follow the discussion 
presented by the authors in reference 23. The 
purpose of this paper is to show the linkage between 
knowledge, creativity, and design, and to relate these 
concepts to the best practices of multidisciplinary 
design teams. The paper will first discuss the 
environment within the aerodynamic design 
community. This will be followed bv a review ol the 
concepts of design, creativity, and knowledge. A 
discussion of creative teams and the creative potential 
concept will follow. The final section of the paper 
will discuss future creative organizations and work 
environments. 

To assist the reader a listing of definitions is provided 
after the references. 

PROBLKM AND SOLUTION PATHWAY 
Problem 

One of the significant issues facing the engineering 
community is the failure to recognize that a designer 
is a unique and highly skilled individual. The 

distinction between a designer and a person doing 
design work is vast. The first uses intellect, 

knowledge, and passion to create, and the latter 
executes a routine process that re-creates. 

In today's environment the issue of cost is the 
dominate force in design 23 . This focus is often at 
the expense of creativity, innovation, and 

performance. The primary means to achieving cost 
reduction has been through the use of computer 
modeling of design processes 24 . While computers are 
an important tool to the engineering design 

community, it is a mistake to view them as the 
primary tool of design. In the quest to be viewed as 
being on the forefront of technology, may be looking 
to automate processes, and computerize the 

capabilities and skills of engineers"' 117 . In 
developing these engineering information models, 
design knowledge is frozen at a given time and new 
knowledge generated some time in the future w ill not 
be utilized. The concern is that the creative and 
innovative practices of the past will be replaced with 
routine processes. Another possible byproduct of this 
trend is that there will be a severe lack of 
"understanding" of final design performance and 
"judgement" by those doing design to interpret the 
results. If this occurs costly redesign during the 
development and production activ ities could result. 


The final issue of concern is reflected in the design- 
development time line as shown in figure 1. The 
figure shows that the opportunities to impact 
performance or cost of a design are heav ily weighted 
toward the conceptual design phase where the design 
activity is typically guided by first order effects and 
simple models 34 ,K . Very often only a fraction of the 
available knowledge (Design Input) for the particular 
design problem is used during this phase of the 
design. This inverse relationship between knowledge 
used and the ability to impact the design must be 
corrected if significant improvements are to be 
achieved. 


Solution Pathways 

If the reader accepts the notion that knowledge- poor 
design activities are a problem for the aerospace 
community then it is the authors' opinion that the 
pathway to the solution leads through the concepts of 
knowledge and creativity. First, it must be 
recognized that the design activity must be a highly 
creative endeavor if cost is to be minimized and 
performance maximized. Second, the design activity 
must be a knowledge- based activity that will provide 
significant intellectual freedom. And third, designers 
and not the design process are the critical element in 
design 4 . These ideas are graphically depicted in figure 
2, in which the change in the design output 
(performance and cost) as a result of increasing the 
knowledge and creativity, in the early phases of 
design, is shown. By inserting more knowledge and 
creativity into design, greater cost and performance 
benefits will be achieved earlier in the design. The 
result is a better, faster, and cheaper design. 

An extension of these ideas leads to the concept of 
creative design, where a creative design is defined to 
be: a new concept based upon a non-obvious solution 
to a difficult problem. The creative aspect of the 
design is derived out of inspiration that is driv en by a 
difficult challenge. It is important to note that a 
creativ e design requires high degrees of knowledge and 
inspiration, coupled with a very difficult challenge. 
This concept is graphically depicted in figure 3 which 
shows a simple model of a design space defined by 
three parameters: challenge. knowledge, and 

inspiration. As shown in the figure a "traditional 
design" is driven solely by knowledge and challenge 
and a "creative concept" is produced when 
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knowledge and inspiration are used. Also note; 
’’brainstorming" uses inspiration and challenge but 
does not require extensive use of knowledge. But to 
achieve a creative design, one must utilize knowledge 
and inspiration directed towards a difficult challenge. 

FROM DAI A TO DHSKiN 

Data is the foundation of knowledge. Knowledge is 
the foundation of creativity. Creativity is the 
foundation of design. These simple statements would 
lead one to believe that the relationship between data 
and design is straightforward and thus can easily be 
modeled. However, a close examination of the 
transformations of data to know ledge, know ledge to 
creativity, and creativity to design shows that there is 
ever increasing complexity involving both explicit as 
well as implicit assumptions that are made by the 
designer. 'Ib provide additional insight, a simplistic 
model of these relationships is depicted in figure 4. 
Note that the model consists of a closed loop element 
entitled "Research" which represents activities that 
support the development of knowledge, 
understanding, and judgement. In general, the 
activities that produce data and information are 
process driven, whereas knowledge, understanding, 
and judgement are developed by exercising "best 
practices". 

To define knowledge, as related to engineering, one 
must also define data and information. The follow ing 
definitions are prov ided: 

Data: a group of fact or statistics that have not 

been assigned meaning. 

Information: data that has been assigned meaning. 

Knowledge: the sum of vv hat has been perceived and 
learned that allows for the generation of 
new information. 


Knowledge 

A graphical depiction of the interrelation between 
data, information, and knowledge is presented in 
figure 5. The figure shows that data are operated on 
by know ledge to produce information and a new set of 
knowledge. This knowledge set is then used in the 
interaction with new data and new information sets. 


Another important aspect of knowledge creation is the 
four modes of knowledge creation introduced by 
Nonaka" in 1995. see figure 6. In this model, there 
are two basic types of knowledge, tacit and explicit. 
Nonaka states that knowledge is created by the 
conversion of tacit to both tacit and explicit and from 
explicit to both tacit and explicit. 

The final knowledge concept provided herein is the 
notion of a "Knowledge Matrix" (figure 7). which is 
borrowed from the work of Margolis in 1987 on a 
cognitive model he termed the "Belief Matrix". 
Margolis stales that there is tension between critical 
and intuitive scrutiny to a question and the yes (+), 
no (-). and maybe (o) answers to the critical and 
intuitive question form nine affective slates. For this 
discussion, the nine affective states of Margolis have 
been renamed to reflect levels of knowledge 
awareness. In the "Know ledge Matrix", there are four 
levels of knowledge awareness; Known Knowledge. 
Unknown Knowledge. Probable Knowledge, and 
Possible Knowledge. These states relied the concept 
that "all" that is possible to be known can be known. 
For the present discussion Known Knowledge is 
renamed Know ledge, Unknow n Knowledge is renamed 
Understanding. Probable Knowledge is renamed 
Judgement, and Possible Knowledge is renamed 
Inspiration. The definitions of Understanding. 
Judgement, and Inspiration are prov ided below: 

Understanding: identification of the significance for 

certain data, information, and 
knowledge. 

Judgement: the selection of a course of action, based 
on knowledge and understanding. 

Inspiration: the action or power of moving the 
intellect or emotions to impart 
know ledge to achiev e an end goal. 

The important aspect of this concept is that 
traditional design systems that are computationally 
based either use only the C+/I+ Knowledge stale or 
all of the C+ slates. But in a creative design activity 
Knowledge, Understanding, Judgement, and 
Inspiration are used (i.e. all aspects of knowledge) 

A further review of figure 4 show that a person’s 
knowledge, understanding, and judgement form the 
basis for exercising ones creativity. When addressing 
a particular problem, creativity is driven by 
inspiration. When a difficult challenge is presented to 
the designer, creativ ity drives the solution. 
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Design Models 

In contrast to a person doing design, the designer 
begins a "creative design" challenge by taking an 
unbounded view of the problem. This first step is 
critical and may be viewed as designing the design. 
The act of designing the "creative design" provides 
context and content but does not constrain or inhibit 
the activity. The relationship between a "traditional 
design" space and a "creative design" space is depicted 
in figure 8. The creative design space is defined by 
ever-expanding layers of possible solutions that arc 
based in knowledge, understanding, judgement, and 
inspiration. These layers allow for the inclusion of 
unseen solutions to the design challenge. In contrast, 
the graphic of figure 8 represents that typical design 
space as a square box located within the creative 
design space. The box represents the rigid and 
restrictive nature of the traditional design activity that 
does not allow for unknow n solutions to be achieved. 

The operational characteristics of the traditional 
design process and creative design practice are shown 
in figure 9. As one would expect, the true designer 
usually prefers the creative design practice as depicted 
on the right side of figure 9. However, politics, 
resources, and culture attempt to drive the designer to 
the typical design process as shown on the left. The 
major differences between the two are that the 
traditional design process is an inside-out activity, 
which utilizes only a small section of the design 
space, whereas the creative design practice is an 
outside-in activity, which utilizes the full design 
space. 

A Traditional design is initiated by defining a specific 
goal, constraints, selection of tools, and the 
development of a rigid design process (see left side of 
figure 9). The traditional design activity then cycles 
within the framework of the pre-selected design 
prtxrcss to achieve a solution. 

The creative design activity, shown on the right side 
of figure 9, relies upon the challenge and inspiration 
to drive the development of possible solutions while 
basing all decisions on knowledge, understanding, and 
judgement. The creative design practice does not tie 
itself to a goal or constraints, but instead uses the 
goal and constraints to define the challenge. The 
creative design practice does not tie itself to specific 
tools or processes, but relies upon knowledge, 
understanding, and judgement to explore solution 
pathways and candidate designs. 


RECENT EXPERIENCES 


The previous sections of the paper have addressed the 
concepts of design, creativity, and knowledge. This 
section of the paper will provide an aerodynamicisfs 
view of these concepts and relate them to present 
design activities at NASA I^angley Research Center 
(LaRC). 

As mentioned prev iously, there has been a significant 
growth in the use of computers in the design of 
aircraft, and consequently, there has also been an 
equally dramatic reduction in the knowledge and 
creativity used in aircraft design. The changes have 
been significant. It may be argued that we are about 
to realize the 1987 prediction 26 

"... what / see in the year 2020, there will be no wind 
tunnels .... we would be at the point where airplanes 
could be designed by rather low-paid technicians . " 

Dr. Doug Dvvoyer 

At this time, the aircraft design environment is 
focused on computational efficiency as a means to 
reduce cost. And because of the change in the design 
environment, aerodynamicists have a reduced role. If 
this trend continues large amounts of design potential 
may be lost by not including the designers knowledge 
and creativity into the conceptual design activity. 

One reason for these trends in design has been a 
noticeable transition in the work product at NASA 
LaRC. This is especially evident in the field of 
Aeronautics. As depicted in figure 10, there has been 
a transition from a problem-focused research 
organization into a product-development organization. 
Mapping the work prior to 1990 into the technology 
development template (left side of figure 10) shows 
that the work covered the areas of Science. 
Engineering, and Technology Design. However a 
mapping of post 1990 work shows that the areas of 
Science, Engineering, Technology Design, 
Marketing, and Service are now included. While on 
the surface and when viewed in isolation this might 
not appear to be a significant issue. However, when 
one takes into account that over the same period of 
time the budget and staffing have undergone 
significant reductions serious concerns are raised, if 
only Aeronautics is considered, the reductions, as a 
percent of 1990 levels, are much greater. 
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Creative Design Demonstration Activity 

A team of researchers at NASA Langley Research 
Center have recently completed a design activity 
focused on demonstrating new work practices and 
changes to the present work environment which are 
more conducive to a creative research culture. The 
design activity explored ways to introduce knowledge 
and creativity into the research and design 
environment at this (’enter, w ithin the context of a 
national product dev elopment activ ity. The follow ing 
discussion will briefly review the elements of this 
activ ity, however specific details of the activity will 
not he presented due to the sensitive nature of the 
work. Note, a creative design requires inspiration, 
creativity, and knowledge to he present. A 
description of those aspects of this particular activity 
are described below. 

The inspiration for the subject activity was threefold; 

1 ) to remind us of the power of the creative process, 

2) to encourage change in the current research 
environment, and 3) the technical challenge did not 
have an obvious solution. Without the third 
inspiration element the activity would not have 
occurred. In fact, the technical challenge served as the 
vehicle that carried the first two inspiration elements. 

The technical challenge facing the creative design 
team were: 

Improve performance a minimum of 20 r ^ in 
several areas. 

Target design was a new product within the 
detailed design/development phase. 

Team had a three month design window. 

Solution required the development of new 
science, technology, and concepts. 

To address this challenge, a creative design team was 
formed that was composed of individuals with 
appropriate levels of knowledge and skills and with a 
strong desire to actively participate. The first team 
activity was to develop a set of team rules that were 
clear, concise, unambiguous, and >et flexible to allow 
the team to focus on results and not a process. The 
rules selected are listed below: 

- This is a DESIGN ACTIVITY! 
Minimi/e/eliminate reviews 
Minimize/eliminate oversight 
Dedicated discussion space. 

Virtual work environment. 

Daily face-to-face design activity. 

Use "Creative Team" model. 

Enforce the "Sunshine Policy" 


Each of the listed rules contributed equally to the 
success of the team, and all are self descriptive with 
the exceptions of "Creative Team" and "Sunshine 
Policy" which are described below. 

In this case a "Creative learn" is defined as 
group of individuals with a shared 
inspiration directed towards a difficult 
challenge, w ho work in a shared 
environment governed by tacit and explicit 
knowledge based criteria as w ell as the rules 
listed abov e. A more detailed discussion of 
this concept will be presented later in the 
paper. 

The "Sunshine Policy" requires that all 
work, decisions, results, communications, 
etc. be open and available in real time to all 
team members. 

The team achieved success in all aspects of their 
work. The team operated for a 3 month period 
utilizing less than 2 work years of effort and satisfied 
all technical objectives within the resources allocated. 
All of the expectations of both the People Managing 
Work (PMWs) at NASA LaRC as well as the 
customer w ere exceeded by the team. In addition, the 
team was able to deliver new science, technologies, 
concepts, and innovations to the aerospace 
community 

As a result of this demonstration activity, several best 
practices have been identified that should help other 
creative design activities. These best practices are 
listed below: 

People Doing Work (PDWs) must lead 

PDWs must want to activ ely and enthusiastically 

participate 

Use the "Creative Team" model 
Enforce the "Sunshine Policy" 

Eliminate external (management) reviews during 
the design period 

Detailed project plans not required for success 
Management (PMW) function is to provide all 
necessary resources 

Use Hear, Concise, Consistent, and Challenging 
(C‘> goals 
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Creative Teams 

Perhaps the most difficult issue faced by the team was 
the creation of the optimum work env ironment. In 
pursuit of this matter the activity considered a variety 
of models as depicted in figure 1 1. The figure show 
three types of possible teams; Bureaucratic, 
Hierarchical, and Creative. The Bureaucratic team is 
formed by the organization and exists as an element 
of the organization. This team is recognized as being 
in competition with other elements of the 
organization and must operate within the 
organizational policies and culture. The Hierarchical 
team is formed by the participating elements of the 
organization to perform a task of mutual benefit. 
This team is supported as long as it is recognized as 
adding value to the individual elements of the 
organization. Those who have a shared inspiration 
and are committed to achieving success form the 
Creative team. The Creative team operates within the 
organization environment; however, the Creative 
team is not one that is typically supported by an 
organization. The characteristics of a Creative team 
are increased communications, productivity, and 
reduced complexity and cost compared to the other 
two teams. A more detailed description of the 
"Creative Team" model is presented in figure 12. 

The ability to create the preferred environment 
requires the recognition that knowledge is the most 
valued commodity and that creativity and innovation 
are critical. If these three characteristics are present 
then a team can achieve success. 

( RHATIVH POTHNTIAL MODEL 

The success of the creative design team activity gave 
rise to a desire by the authors to communicate this 
experience in a straightforward and managed approach. 
The hope is that through this effort the scientific and 
creative research and technology development 
community will benefit. 

A review of the literature on creativity and teaming 
shows many studies have been conducted on teaming 
and creativity and that simplistic models have been 
created’ 7 ** 1 . However there was not a model 
specifically directed towards research organizations or 
a model that provided an explicit representation of 
creativity. It became apparent that the community 
may benefit from such a model if it were shown to be 
a useful representation. As a result, the concept of 
creative potential was formulated and assessed with a 


survey of a small portion of the aeronautics research 
staff at NASA Langley Research Center. A summary 
of these results are presented below. 

In order to mature the creative potential concept, a 
hypothesis was developed and modeled. Data were 
obtained to evaluate the hypothesis that was then 
updated and revised. The hy pothesis is as follows: 

The hypothesis: 

All people have the potential to be creative 
A person’s creative potential can be both nurtured 
and destroyed by the environment and the type of 
work. 

The creative potential of a research scientist or 
engineer in their field of study is primarily a 
function of an individual's knowledge. 

The knowledge of an individual in a topic area is 
a function of the number of measurable 
contributions in the topic area and the number of 
significant activ ities executed by that person. 

An organization's creative potential is a function 
of the creative potential of the people in the 
organization, the work environment, and the ratio 
of people managing work (PMWs) to people 
doing work (PDWs). 

A person’s creative contribution is a function of 
their own creative potential and the willingness 
of the organization to recognize and facilitate the 
use of their creative output. 

An organization's creative contribution is a 
function of the organizations creative potential 
and the willingness of the organization to accept 
failure. 

The above hypothesis was formulated into a variety 
of sub- parameters, which were then rolled up into the 
four primary parameters; Personal Creative Potential 
(PCP), Organizational Creative Potential (OCP), 
Personal Creative Contribution (PCC), and 
Organizational Creative Contribution (OCC). 

PCP reflects the creative potential of a single PDW 
and is defined as a function of a PDWs background 
and experiences (knowledge) and work environment as 
derived from the responses in tables 2 and 3, 
respectively. The PCP term is formulated to vary 
between 0.0 and 1 .0. 

The OCP term represent an organizations creative 
potential and as such is defined by the average PCP 
of the organization's PDWs, the ratio of PMWs to 
PDWs, and the organization’s work environment. 
The information for this parameter is contained in 
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tables 2, 3, and 4. The OCP term is formulated to 
vary between 0.0 and 1 .0. 

PCC represent the ability of a person to utilize their 
PCP and produce creative products. The PCC term is 
primarily a function of PCP and OCP and as such is 
hascd upon the data contained in tables 2. 3, and 4. 
The PCC term is formulated to vary between 0.0 and 
1 . 0 . 

Finally; OCC is a function OCP and the 
organization's acceptance of risk. The organization's 
acceptance of risk is based upon the information 
contained in table 5. The OCC term is formulated to 
vary between 0.0 and 1 .0. 

For each of these primary parameters a creativity 
threshold or target value was identified which defines 
the probability that a creative design output will 
result from the work effort. A preliminary analy sis of 
the primary parameters determined that the creativity 
threshold values would be 0.75 for PCP and OCP. 
and 0.50 for PCC and (XT'. The threshold values for 
PCP and OCP reflect an overall objective of an 
organization that embraces creativity and that the 
majority of the staff should be performing creative 
work; thus a potential value (i.e.. PCP and OCP) 
should be much greater than .50. The threshold 
values for PCC' and ()('(' reflect the fact that they are 
a product of OCP and PCP and thus should be set at 
a value approximately equal to the product of the 
threshold values for PCP and OCP (i.e., 0.75 \ 
0.75). 

To evaluate the model, data were obtained through a 
survey of the aeronautics PDW staff at NASA 
l^ngley Research ('enter (LaRC). The survey 
contained 48 questions distributed between four 
categories as dvx'umented in table I . The lour survey 
areas were; work elements, background and 
experiences, team experiences, and organizational 
experiences. 

The survey was distributed to 50 PDWs who work in 
aeronautics at NASA LaRC. The individuals that 
participated in the survey represent the bnoad 
spectrum of aeronautics work at this ('enter. Included 
in the 50 PDWs surveyed were the 10 PDWs 
involved in the creative design team activ ity discussed 
previously. For the following discussion responses 
from the creative design team PDWs w ill be referred 
to as CDT PDWs and the aeronautics PDW 
responses, which include the CDT PDWs, will be 
referred to as IxtRC PDWs. The survey sample 


represented a little over 14% of the aeronautics PDW 
staff at this ('enter. A total of 34 surveys were 
returned for analysis of which 8 were from CDT 
PDWs. A copy of the survey and statistical 
information of the responses are presented in tables 2, 
3, 4, and 5 for both the LaRC PDWs and the CDT 
PDWs. Iable 2 contains information related to the 
background and experiences of the respondents (6 of 
the 48 questions), tables 3. 4. and 5 contain 
information on the remaining 42 questions, and table 
6 contains information on the four computed primary 
parameters (i.e., PCP. OCP, PCC, and OCC). 

Analy sis of the survey results w as performed for both 
the LaRC PDW responses and the CDT PDW 
responses in order to show the influence of work style 
and work environment on the creative potential 
model. All of the results are presented in figures 13 
through 18. 

As mentioned previously. PCP is a function of a 
person's knowledge and work environment, see 
figures 13 and 14 respectively. For this activity, a 
person's knowledge is measured by the tangible and 
measurable work products as a function of years in 
research. As shown in figure 13. the knowledge 
elements were; invention disclosures, technical 
reports, experimental studies, computational studies, 
and design studies. Results for both the LaRC PDWs 
and the subset of CDT PDWs are presented for 
comparison. Overall, the CDT PDWs produced more 
know ledge products than the LaRC PDWs for a given 
number of years of work experience. 

Presented in figure 14 are results obtained for the 
work environment of a PDW. As was done in figure 
13, results are shown for both the LaRC PDWs and 
the subset of CDT PDWs. The survey questions 
addressing work environment were in a matrix format, 
in which respondents were asked to distribute their 
time spent performing creative work, non creative 
work, creative thinking, learning, lost time mistakes, 
and overhead for their usual work, team work, and 
their view of the ideal work env ironment. Note, the 
responses obtained for team work from the CDT 
PDWs was with respect to the creative design team 
while the results obtained from the LaRC PDWs was 
to represent their typical team experiences. An 
ov erview of the results show that the subset of CDT 
PDWs spend more lime doing creative work and 
creative thinking and less time doing non-creative 
work, learning, lost lime mistakes, and overhead, 
compared to the overall group of LaRC 14) Ws. This 
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indicates that the CDT PDWs have been able to 
fashion a work environment that is more conducive to 
creativity despite being in the same organization as 
the other PDWs sun eyed. A comparison of the 
IxtRC PDWs and CDT PDWs change in percent time 
between the team results and the ideal results show 
smaller increments for the CDT PDWs, indicating 
that the creative team experience more closely 
matched their preferred work environment. 

A comparison (PCP values for the usual, team, and 
ideal work experiences) of the LaRC PDW and CDT 
PDW is shown in figure 15. These results show that 
the LaRC PDW values never achieve the threshold 
value of 0.75, even for the ideal case. The results for 
the CDT PDWs show that the threshold value was 
reached for both the team and ideal work 
environments. The failure of either of the usual work 
environment results to achieve the threshold value is 
likely due to the large amounts of overhead and non- 
creative work loads for all PDWs surveyed. 

The OCP results are shown in figure 16. These 
results are presented in the same format as that for 
PCP above. Recall that OCP is a function of the 
PCP value, ratio of PMWs to PDWs, and the 
organization's imposed work environment. The 
formulation of this parameter allows for OCP to 
exceed PCP: however, that did not occur for these 
results. An analysis of this parameter for the usual 
organizational work environment showed that due to 
the large PMW to PDW ratio of 0.4 (extracted from 
NASA LaRC’ 2000 phonebook. 1 PMW for every 2.5 
PDWs), the OCP value was consistently lower than 
the PCP value. For the team values, the PMW to 
PDW ratio was set to 0.10 (anecdotal evidence from 
survey showed that a traditional team consisted of 10 
researchers with one lead). For the ideal case, the 
ratio was set to 0.0025 (in order to minimize the 
impact of the PMW to PDW ratio, we needed to 
minimize the ratio, so 1 PMW to 400 PDWs was 
chosen). A review of the OCP data shows similar 
trends as that seen for PCP but at a slightly reduced 
level. Note that only the ideal CDF PDW results 
achieve the threshold value for creativity. 

The next two parameters, PCC and OCC, are used to 
measure the ability of a creative output to contribute 
to knowledge growth or technology development. 
Note, the PCC and OCC values are both a function 
of OCP and either PCP or organizational risk, 
respectively and as a result the creativity threshold 
values for both the PCC and OCC parameters is set 
at 0.50. These threshold values reflect the 


constraining nature of an organization on creativity. 
As defined, an organization is the executive structure 
of a business. Creativ ity requires knowledge and the 
ability to use that knowledge with emotions to 
achieve an end goal. Based on these definitions, it is 
clear that an organization benefits from creativity, 
however, creativ ity does not need an organization to 
flourish. 

A rev iew of both the PCC results, see figure 17. and 
the OCC results, see figure 18, show results similar 
to those seen for PCP and OCP. in general the CDT 
PDW results are higher than the I.aRC PDW results 
and all show an increasing level moving from the 
usual work environment to the team work 

environment and finally to the ideal work 

environment. Note that none of the OCC results 
achieve the threshold value due to the perceived 
inability of the organization to accept any significant 
risk as noted in the survey responses. 

The survey results obtained support the creativity 
potential model proposed. Based upon this very 
preliminary analysis, it may be concluded that the 
creative potential model may be a useful tool in 
assessing and developing creative work environments, 
however, it is recognized that additional assessments 
are needed to further refine this model. 

FUTURHDIRHCTIONS 

Based upon the observations presented above, it is 
apparent that for an organization that desires to 
embrace creativity as an essential element and core 
value needs to provide the appropriate environment 
for those elements to exist and flourish. The focus 
should be to create an environment that facilitates 
creativity, and knowledge generation, and fosters 
inspiration. Creativity is an especially important 
element for a research organization. It must also be 
recognized that the realities of doing business in a 
cost driven world dictate that the entire organization 
can not be allowed to operate on creative principles. 
But for the success of a research organization at least 
a portion of the staff must have the freedom to pursue 
creative solutions to problems and as a result not be 
overly constrained by the institutional aspects of that 
organization. 

The first step is to implement the creative team 
environment as described in figure 19. The creative 
team environment challenges leaders to foster 
inspiration in order to facilitate creative work directed 
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at a difficult challenge within a shared environment. 
The creative team is characterized by unconstrained 
knowledge based communications and knowledge 
based decisions. The creative team environment 
does not replace the typical (structured) organization, 
but clouds the typical organization to facilitate a 
creative environment (see figure 20). 

The creation of this environment requires the 
optimization of management staffing ratio (PMW to 
PDW ratio) as shown in figure 21 and the acceptance 
of a knowledge growth model as shown in figure 22. 
Both of these changes are required if success is to be 
achieved. 

The creativity potential model is a function of the 
PMW/PDW ratio. In order to assist the reader in 
implementing a creative organization based upon the 
model, the authors would like to propose the 
following method. To determine the PMW/PDW 
ratio for a particular engineering/scienlific 
organization, graph the organizations percent of work, 
based upon resources expended, against work type that 
the organizations performs, see blue curve on figure 
2 1 .The percent work curve is a representation of the 
distribution of work type within an organization. 
Also presented in figure 21 is a representative curve 
of the PMW/PDW ratio as a function of the work 
type for scientific/engineering organizations, see red 
curve. The PMW/PDW ratio curve reflects 
representative PMW/PDW values for organizations 
with work that is characterized by the work type 
listed. The PMW/PDW ratio data was obtained from 
phone conversations of scientific and engineering 
organizations and laboratories. In the most simplistic 
analysis the optimum PMW/PDW is determined by 
the intersection of the % Work moment of inertia for 
an organization and the PMW/PDW curve, as 
depicted in the figure. The information presented 

indicates that if a research organization is focused on 
creativity it should minimize the PMW/PDW ratio. 
If an organization were focused on product 
development/markeling, then it would typically have 
a large PMW/PDW ratio. 

The second critical activity is the implementation of a 
knowledge growth mechanism for the organization. 
Note that this model should be used by each 
indiv idual, team, organization element, as well as the 
complete organization. A variation-selection model, 
discussed in reference 36, that serves this purpose is 
presented in figure 22. The model shows that there 
are three critical elements in knowledge growth: (1) 
introduction of variations, (2) consistent selection 


practices, and (3) preservation and propagation of the 
variations. 

The introduction of variations is the means in which 
the seeds of a creative output are laid. The variation 
inputs must have an unseen/unknown output if they 
are to be of benefit. The concept of unknown output 
is critical because a creative output is unknown until 
it is manifested. The second step is the use of 
consistent selection practices that are not constrained 
to rigid processes, but utilize both tacit as well as 
explicit knowledge. An example would be the 
stopping point of a wing design. This is hased upon: 
how it looks, how it performs at the design point, 
how it compares to previous w ork. and the projection 
of its performance over the entire flight envelope. The 
final step is preservation and propagation of the 
variations in which the data and information air 
converted to knowledge, understanding, and judgement 
through documentation and sharing mechanisms. 

CONCLUDING RHMARKS 

Design is a greatly misunderstood and undervalued 
practice within the engineering comm unit}. It is 
important to recognize that the design philosophy 
not the design process, defines the design space. The 
design philosophy, which is developed by the 
designer, must not be constrained b\ known rules, 
constraints, or by computational tools. It must be 
recognized that the ability to perform conceptual 
design must not be contingent upon the ability to 
obtain a computational solution. The efficient*) and 
accuracy of the conceptual design phase is directly 
related to the know ledge used, and thus we must focus 
on including ever-greater amounts of knowledge and 
creativ ity into the conceptual design phase. 

To address the need for improved design capability the 
following recommendation are offered: 

• Implement the creative team model to 
facilitate the development of design 
knowledge, design skills, and design 
systems. 

• Utilize the creative potential model as a 
means to measure and change the design 
environment. 

• Recognize that design is a knowledge-driven 
creative activity, not a computational 
process. 
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DEFINITIONS 

Communications 

Interchange of thoughts, opinions, or ideas 

Complexity 

Composed of confusing interrelated parts 

Create 

To bring something new into being. 

Requires knowledge and inspiration. 

Creative Design 

To bring into being a new concept based upon a non- 
obvious solution to a difficult problem. 

Requires knowledge, inspiration, and a challenge. 

Creative Team 

Individuals with a shared Inspiration directed towards a 
difficult Challenge who work in a shared environment 
governed by tacit and explicit knowledge based 
criteria. 

Culture 

Validated basic assumptions developed by a given 
group and taught to new members as the correct way 
to perceive, think, and feel. 

Challenge 

A difficult problem that arouse or stimulate creative 
effort. 


Data 

A group of facts or statistics that have not been 
assigned meaning. 

Design 

A new concept, model, or artifact with desired 
function, performance, or end. 

Requires knowledge and a challenge. 

Environment 

Aggregate of all conditions and influences affecting an 
individual or group. 

Epistemology 

The stud) or a theory of the nature and grounds of 
knowledge especially with reference to its limits and 
validity. 

Excite 

To increase the activity of a group or individual. 
Honesty 

Integrity and straightforwardness in ones conduct, 
thought, and speech. 

Honor 

A fastidious allegiance of the standards of ones 
profession 

Information 

Data that has been assigned meaning. 

Innovation 

The act of introducing something new. 

Inspiration 

The action or power of moving the intellect or 
emotions to impart knowledge to achieve an end goal. 

Judgment 

The selection of a course of action, based on 
knowledge and understanding. 

Knowledge 

The sum of what has been perceiv ed and learned that 
allows for the generation of new information from 
new data. 

Leadership 

The art of influencing other people to achiev e shared 
goals. 

Learn 

To gain knowledge or understanding. 

Management 

The collectiv e body who conduct, control, and direct 
an interest. 
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Method 

An orderly procedure followed to achieve an end. 

Model 

A simplified representation of a concept or system in 
which all critical features are described. 

Motivate 

To provide an idea a need or to stimulate an emotion 
to incite action. 

Organization 

The executive structure of a business. 

Policies 

General guidelines for making management 
decisions. 

Practice 

The accepted application of knowledge within a 
professional discipline to achieve an objective. 

Principles 

Simple and direct statements of basic beliefs. 

Procedures 

Instructions defined by management for performing a 
sequence of actions. 

Problem 

A question proposed for solution. 

Productivity 

Effective in bringing forth originative and creative 
action 


Process 

A series of specific actions or operations producing a 
known end state. 

Research 

Critical and exhaustive investigation aimed at 
developing new understanding through the acquisition 
of new knowledge. 

Results 

Something obtained by investigation. 

Team 

People who are committed to solving a shared 
problem. 

Technology 

A capability given by the practical application of 
knowledge. 

Understanding 

Identification of the significance for certain data, 
information, and knowledge. 

Values 

Basic convictions that a specific mode of conduct or 
end-state of existence is preferable to an opposite 
mode of conduct or end-state existence. 

Virtual Teams 

Teams whose primarily means of interaction is 
electronic. 

Work 

Exertion of faculties to accomplish something to 
solve a problem. 
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A Creative Design is a new concept based upon a 
nonobvious solution to a difficult problem. 

A Creative Design requires knowledge, inspiration, 
and a challenge. 



Figure 3. Design space model for creative design. 
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Explicit - clearly developed meaning, objective 


TACIT 

FROM 

EXPLICIT 

Figure 6. Knowledge creation model. 
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Figure 7. Knowledge matrix. 
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Figure 9. Traditional design process and creative design practice models. 
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Figure 10. technology development model. 
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Figure 11. Team types. 
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Creative Team: Individuals with a shared Inspiration directed 
towards a difficult Challenge who work in a shared environment 
governed by the criteria listed below. 


Only accept work with Clear, Concise, 

Consistent, and Challenging (C 4 ) goals. 
Only Knowledge Based decisions. 


Organization 

Environment 


• Sunshine Policy in effect. 

• Dedicated discussion/debate space. 

• Virtual work/knowledge sharing 
environment. 

• Daily face-to-face design activity. 

• All resources provided at start. 

Figure 12. Creative team model. 


Minimize/Eliminate reviews. 
Minimize/Eliminate oversight. 
Minimize/Eliminate planning. 



All members want to actively 
participate. 

All members perform 
technical work. 
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Figure 13. Background and expereince survey results. 
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(b) Non-Creative Work 

Figure 14. Work elements survey results. 
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(d) Learning 


Figure 14. Continued. 
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(e) Lost Time Mistakes 
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(f) Overhead 


Figure 14. Concluded. 
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Figure 15. Personal Creative Potential (PCP) survey results. 
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Figure 16. Organizational Creative Potential (OCP) survey results. 
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Usual Team Ideal 


Figure 17. Personal Creative Contribution (PCC) survey results. 
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Figure 18. Organizational Creative Contribution (OCC) survey results. 
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Leaders foster Inspiration to facilitate Creative work directed 
towards a difficult Challenge within in a shared environment 
governed by the criteria listed below. 


• Work with goals. 

• Knowledge Based decisions. 


Minimize/Eliminate program reviews. 
Minimize/Eliminate program planning. 



Sunshine Policy in effect. 

Dedicated discussion spaces. 

Virtual knowledge sharing environment. 
Honor resource promises. 


• All members want to enthusiastically 
participate. 

• All members perform technical work. 

• All members contributions 
documented. 


Figure 19. Creative environment model. 



Figure 20. Creative organization model. 
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Figure 21. Creative staffing optimization. 
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Figure 22. Knowledge growth model. 
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Data has been obtained in the following 4 categories using a 


total of 48 questions. 


• Work Elements: 

(1) Type of Work 
(Creative, Non-Creative) 

(2) Thinking 

(3) Learning 

(4) Overhead 

• Background and Experiences: 

(1) Years in Research 

(2) Type of Products 

(3) Number of Products 

• Team Experiences: 

(1) Goal Management 

(2) Work Environment 

(3) Member Contributions 

(4) Decision Mechanisms 

• Organizational Experiences: 

(1) Goal Management 

(2) Work Environment 

(3) Acceptance of Risk 

(4) Decision Mechanisms 

(5) Recognition and Reward 


Table 1 . Survey structure. 
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Please give some information about your background and experience by filling 
out the sections below. Estimates are tine if exact numbers are not available. 


Background and Experience 

Number 

Years of work experience in research 


Invention disclosures authored 


Scientific/Engineertng documents authored (NASA papers, conference 
papers, journal articles, books, thesis/dissertation, white papers) 


Experimental studies conducted (involving model prep, testing, data 
acquisition, data reduction, data analysis) 


Computational studies conducted (involving geometry modeling, surface 
and volume grid generation, code runs, data analysis) 


Design studies conducted (involving definition of design space, 
development of model, analysis, selection of final design ) 



(a) original questionaire 




LaRC 

CDT 

Years of Work Experience 

Average 

20.24 

16.57 

St. Deviation 

9.64 

5.47 

Minimum 

2.00 

S.00 

Maximum 

41.00 

25.00 

Invention Disclosures 

Average 

4.65 

12.00 

St. Deviation 

1 1 .32 

17.S6 

Minimum 

0.00 

0.00 

Maximum 

40.00 

40.00 

Scienti fie/Enginecri ng documents 

authored. 

Average 

37. IS 

43.00 

St. Deviation 

23.60 

23. 13 

Minimum 

1 .00 

IS. 00 

Maximum 

90.00 

SO. 00 

Experimental Studies Conducted 

Average 

18.00 

I4.S6 

St. Deviation 

20. 10 

22.60 

Minimum 

0.00 

0.00 

Maximum 

76.00 

60.00 

Computational Studies Conducted 

Average 

IS. 15 

31.43 

St. Deviation 

23.17 

3 1 .72 

Minimum 

0.00 

10.00 

Maximum 

100.00 

1004)0 

Design Studies Conducted 

Average 

4.21 

7.00 


St. Dev lation 

6.35 

7.72 


Minimum 

0.00 

0.00 

Maximum 

30.00 

22.00 


(b) responses 

Table 2. Statistical information on the background and experiences of individual survey respondents 
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In this section of the survey, please estimate the percentage of time you spend 
doing each of the work elements listed below. Make this estimation for your 
normal work routine, for work spent during the selected team activity, and tor 
your ideal mix. 



Percent of time Spent 

Work Elements 

During 

Usual 

Work 

During 

Team 

Activity 

Your 

Ideal 

Mix 

Creative Work 

Bring into being something new: 
knowledge, data, concepts, or ideas. 




Non-Creative Work 

Work that does not create something 
new r . 




Creative Thinking 

To exercise the powers of judgement, 
conception, and inference 




Learning 

To gain knowledge or understanding 




Lost Time Mistakes 

Repeating work. 




Overhead 

Management and organizational tasks, 
paperwork, or activities. 




Total (should add up to 100) 





(a) original questionaire 



laRC 

cur 


Isual 

Team 

Ideal 

l : sual 

Team 

Ideal 

Creative Work., '7- work effort 
(Bring into being something new; 
knowledge, data, concepts. or 

Average 

23.18 

24.76 

39 22 

33.57 

53 . 57 

55.00 

St. Deviation 

18.26 

19. 48 

17.51 

19.52 

14 92 

18 03 

Minimum 

0.00 

5.00 

15.00 

20.00 

40.00 

30.00 

Maximum 

75,00 

75.00 

80.00 

75.00 

75.00 

80.00 

Non-Creative Work, % work effort 
(Work that does not create 
something new ) 

Average 

25.7V 

22.91 

1 1.32 

17.86 

6.43 

5.00 


16 VI 

17.33 

10.02 

10.75 

3 78 

7.64 


0.00 

0.00 

0.00 

0.00 

0.00 

0.00 


70.00 

60.00 

30.00 

35.00 

10.00 

20.00 

C reative Thinking, % work effort 
(To exercise the powers of 
judgement, conception, and 


10.91 

12 26 

20.8 V 

14.29 

15.71 

18.86 

St. Deviation 

6.06 

7 . V 5 

7 76 

4.50 

8 86 

5.84 

Minimum 

0.00 

0.00 

0.00 

10.00 

5.00 

10.00 


25.00 

30.00 

40.00 

20.00 

25.00 

25.00 

Learning, '7 work effort 
(To gain knowledge or 
understanding.) 


12.53 

14.03 

20.92 

10.00 

1 2.86 

17.57 


8 53 

12.50 

8. 1 1 

2.89 

4 88 

5 13 

Minimum 

5.00 

0.00 

10.00 

5.00 

5,00 

10,00 

Maximum 

50.00 

50.00 

SO 00 

15.00 

20.00 

25.00 

laist Time Mistakes. work effort 

(Repeating work. ) 

Average 

7 12 

8 76 

2.65 

6 43 

7. 14 

0.71 

Si. Deviation 

7.03 

7.28 

5.80 

5 56 

5 67 

1.89 

Minimum 

0.00 

0.00 

0.00 

0.00 

5.00 

0.00 

Maximum 

40.00 

40.00 

30.00 

15.00 

20.00 

5.00 

Overhead, % work effort 
(Management and organizational 
tasks, paperwork, or activities.) 

Average 

20.47 

17.26 

5.00 

17.86 

4.29 

2 86 

St, Deviation 

12.62 

12.65 

5.64 

8.59 

1.89 

3 93 

Minimum 

0.00 

0.00 

0.00 

5.00 

0.00 

0.00 

Maximum 

60.00 

60.00 

25.00 

30.00 

5.00 

10.00 


(b) responses 

fable 3. Statistical information on the Work Element survey. 
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These criteria will be used to evaluate your organization's environment and creative 
potential. Please rate the criteria on a scale of 1 through 10, where 10 indicates "strongly 
agree" and 1 indicates "strongly disagree". In each case, consider the criteria in the 
context of your work and personal experience within your organization. 


CREATIVE CRITERIA 

Rank 1 - 10 

10 = STRONGLY AGREE 
1 = STRONGLY DISAGREE 

Resources available at start of research activities (CC 1 ) 


Intellectual freedom to solve problems (CC2) 


Open communications (CC3) 


Only people doing work (PDW) involved, not managers (CC4) 


Autonomous work environment (CC5) 


Knowledge driven decisions (CC6) 


Simple operation requirements (CC7) 


Simple organization requirements (CC8) 


Goals that are concise, consistent, clear, challenging (CC9) 


Passionate about work (CC 10) 


Flexible work approach (CC1 1) 


Individual and team recognition (CC12) 


Open to failure as well as success (CC 1 3 ) 



(a) original questionaire 

Table 4. Statistical information on the creative criteria survey questions. 
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Creative Criteria 


Resources avaialable at start of 

Average 

research activity . 

St. Deviation 


Minimum 


Maximum 

Intellectual freedom to solve 

Average 

problem. 

Si. Deviation 


LaRC 

5.50 

2.48 

1.00 

10.00 

7.79 

2.01 


CDT 
5.29 
3.40 
I .00 
10.00 
7.43 
1 .90 



Minimum 

2.00 

5.00 

Maximum 

10.00 

10.00 

Open communications. 

Average 

6.97 

6.86 

St. Deviation 

2.35 

2.79 

Minimum 

1 .00 

1.00 

Maximum 

10.00 

10.00 

Only PDW’s involved, not PMW's. 

Average 

5.71 

6.29 

St. Deviation 

2.39 

2.63 

Minimum 

1 .00 

1.00 

Maximum 

10.00 

8.00 

Autonomous work environment. 

Average 

7.38 

8.86 

St. Deviation 

2.41 

1.07 

Minimum 

1.00 

7.00 

Maximum 

10.00 

10.00 

Knowledge drioven decisions. 

Average 

6.32 

7.71 

St. Deviation 

2.18 

1.60 

Minimum 

2.00 

5.00 

Maximum 

10.00 

10.00 

Simple operation requirements. 

Average 

5.44 

7.14 

St. Deviation 

2.29 

l .77 

Minimum 

l .00 

5.00 

Maximum 

10.00 

10.00 

Simple organization requirements. 

Average 

5.09 

6.86 

St. Deviation 

2.39 

1 .77 

Minimum 

1 .00 

5.00 

Maximum 

10.00 

10.00 

Concise, consistent. clear , and 
challenging goals. 

Average 

5.76 

6.00 

St. Deviation 

2.15 

2.52 

Minimum 

1.00 

1 .00 


Maximum 

10.00 

9.00 

Passionate about work. 

Average 

7.15 

8.00 

St. Deviation 

2.39 

2.52 

Minimum 

1.00 

3.00 

Maximum 

10.00 

10.00 

Flexible work appraoch. 

Average 

7.32 

8.00 

St. Deviation 

2.69 

3.21 

Minimum 

1 .00 

1 .00 

Maximum 

10.00 

10.00 

Individual and team recognition. 

Average 

6.47 

6.29 

St. Deviation 

2.42 

2.75 

Minimum 

1 .00 

1 .00 

Maximum 

10.00 

9.00 

Open to fialure as well as 
success. 

Average 

5.47 

6.86 

St. Deviation 

2.3 1 

2.85 

Minimum 

1. 00 

1 .00 

Maximum 

10.00 

10.00 


(b) responses 
Table 4. Concluded. 
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These criteria will be used to evaluate a selected team experience you have participated in 
during the last 5 years. Please rate the criteria on a scale of 1 through 10, where 10 indicates 
"strongly agree" and 1 indicates "strongly disagree". 

Team (optional): . 


TEAM EXPERIENCE (TE) 

Rank 1 - 10 

10 = STRONGLY AGREE 
1 = STRONGLY DISAGREE 

Team goal is understood (TE 1 ) 


Team goal is do-able (TE2) 


Team goal does not change (TE3 ) 


Team members contribute equally (TE4) 


Team members have opportunity for equal input (TE5) 


Decisions are knowledge based (TE6) 


Participation on the team is voluntary (TE7) 


Team members decide their individual work load (TE8) 


All information is shared continually (TE9) 


The team is autonomous (TE 10) 


Team operations and structure are simple (TE1 1 ) 



(a) original questionaire 

Table 5. Statistical information on the team experience survey questions. 
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Team Experience 


Team goal is understood. 


Team goal is do-able. 


learn goal does 

not change. 

Team members 
equally . 

contribute 


Team members have opportunity 
for equal in pul. 


Decisions are knowledge based. 


Participation on the learn is 
voluntary . 


Team members decide iheir 
individual work loads. 


All information is shared continually 


The team is autonomous. 


Team operations and structure 
are simple. 


St. Deviation 


Minimum 


Maximum 


St. Deviation 


Minimum 


Maximum 


St. Deviation 


Minimum 


Maximum 


St. Deviation 


Minimum 


Maximum 


Si. Deviation 


Minimum 


Maximum 


St. Deviation 


Minimum 


Maximum 


St. Deviation 


Minimum 


Maximum 


St. Deviation 


Minimum 


Maximum 


St. Deviation 


Minimum 


Maximum 


St. Deviation 


Minimum 


Maximum 


St. Deviation 


Minimum 


Maximum 


LaRC 


8.06 


2.17 


2.00 


10.00 


7.32 


2.2: 


2.00 


10.00 


7.24 


2.72 


1 .00 


10.00 


.26 


2.09 


3.00 


10.00 


7.83 


2.27 


2.00 


10.00 


7.41 


2.43 


2.00 


10.00 


6.50 


2.98 


1 .00 


10.00 


7.32 


2.64 


1.00 


10.00 


6.9 


2.64 


1.00 


10.00 


6 . 


2.81 


1.00 


10.00 


6.76 


2.92 


1 .00 


10.00 



7.) 

14 

2.1 

19 

5.( 

)() 

10.00 


9.00 


1 .29 


7.00 


10.00 


6.43 


2.44 


3.00 


9.00 


9.86 


0.38 


9.00 


10.00 


9.43 


0.79 


8.00 


10.00 


7.71 


2.75 


3.00 


10.00 


9.29 


0.76 


8.00 


10.00 


9.43 


0.79 


8.00 


10.00 


9.00 


0.82 


8.00 


10.00 


9.00 


0.82 


8.00 


10.00 


(b) responses 
Table 5. Concluded. 
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